State Space Local Linear Prediction

Local linear prediction is one of several methods that have been applied to prediction of real time series including financial time series. The difference from global linear prediction is that, for every single point prediction, a different linear autoregressive (AR) model is estimated based only on a number of selected past scalar data segments. Geometrically, these data segments correspond to points close to the target point when the time series is viewed in a pseudo-state space with dimension equal to the order of the local AR model. The parameters of the local linear model are typically estimated using ordinary least squares (OLS). Apart from potential linearisation errors, a drawback of this approach is the high variance of the predictions under certain conditions. It has been shown that a different set of so-called regularisation techniques, originally derived to solve ill-posed regression problems, gives more stable solutions (and thus better predictions) than OLS on noisy chaotic time series. Three regularisation techniques are considered, i.e. principal component regression (PCR), partial least squares (PLS) and ridge regression (RR). These methods reduce the variance compared to OLS, but introduce more bias. A main tool of this analysis is the Singular Value Decomposition (SVD), and a key to successful regularisation is to dampen the higher order SVD components. For the sake of completeness, truncated total least squares is discussed as well, which is designed to solve “error-in-variables” problems. Even though it would be expected that this method is more appropriate for noisy time series, it turns out to give the worst predictions. This chapter will describe the general features of local linear prediction and particularly the OLS solution and the regularisations. The statistical properties of the methods will be highlighted and explained in the setting of local linear prediction. The superiority of the predictions using regularised solutions over OLS predictions will be demonstrated using simulated data and financial data.